Approximate Contrastive Free Energies for Learning in Undirected Graphical Models
نویسندگان
چکیده
We present a novel class of learning algorithms for undirected graphical models, based on the contrastive free energy (CF). In particular we study the naive mean field, TAP and Bethe approximations to the contrastive free energy. The main advantage of CFlearning is the fact that it eliminates the need to infer equilibrium statistics for which mean field type approximations are particularly unsuitable. Instead, learning decreases the distance between the data distribution and the distribution with onestep reconstructions clamped on the visible nodes. We test the learning algorithm on the classification of digits.
منابع مشابه
Bethe Free Energy and Contrastive Divergence Approximations
Bethe Free Energy and Contrastive Divergence Approximations for Undirected Graphical Models Yee Whye Teh Doctorate of Philosophy Graduate Department of Computer Science University of Toronto 2003 As the machine learning community tackles more complex and harder problems, the graphical models needed to solve such problems become larger and more complicated. As a result performing inference and l...
متن کاملParticle Filtered MCMC-MLE with Connections to Contrastive Divergence
Learning undirected graphical models such as Markov random fields is an important machine learning task with applications in many domains. Since it is usually intractable to learn these models exactly, various approximate learning techniques have been developed, such as contrastive divergence (CD) and Markov chain Monte Carlo maximum likelihood estimation (MCMC-MLE). In this paper, we introduce...
متن کاملlibDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models
This paper describes the software package libDAI, a free & open source C++ library that provides implementations of various exact and approximate inference methods for graphical models with discrete-valued variables. libDAI supports directed graphical models (Bayesian networks) as well as undirected ones (Markov random fields and factor graphs). It offers various approximations of the partition...
متن کاملFastInf: An Efficient Approximate Inference Library
The FastInf C++ library is designed to perform memory and time efficient approximate inference in large-scale discrete undirected graphical models. The focus of the library is propagation based approximate inference methods, ranging from the basic loopy belief propagation algorithm to propagation based on convex free energies. Various message scheduling schemes that improve on the standard sync...
متن کاملStructure Learning of Undirected Graphical Models with Contrastive Divergence
Structure learning of Markov random fields (MRFs) is generally NP-hard (Karger & Srebro, 2001). Many structure learners and theoretical results are under the correlation decay assumption in the sense that for any two nodes i and k, the information about node i captured by node k is less than that captured by node j where j is the neighbor of i on the shortest path between i and k (Netrapalli et...
متن کامل